From Images To Animation Models

We show that we can effectively fit complex animation models to noisy image data. Thus, complete head models can be acquired with a cheap and entirely passive sensor. The shape and motion parameters of limbs can be similarly captured and fed to existing animation software to produce synthetic sequences. In this paper, we show that we can effectively fit complex facial and body animation models to calibrated or uncalibrated sets of images and video sequences. We need minimal human intervention and neither targets nor structured light nor any other active device. We first introduce our animation models and then describe the fitting techniques we have developed. Animation Models The modeling and animation of Virtual Humans has traditionally been decomposed into two subproblems: facial animation and body animation. In both cases, muscular deformations must be taken into account, but their roles and importance differ. Facial animation primarily involves deformations due to muscular activity. Body animation, on the other hand, is a matter of modeling a hierarchical skeleton with deformable primitives attached to it so as to simulate soft tissues. It is possible to model bodies in motion while ignoring muscular deformations, whereas to do so for facial animation is highly unrealistic. For heads, we use the facial animation model that has been developed at University of Geneva and EPFL and is depicted by Figure 1(a). It can produce the different facial expressions arising from speech and emotions. The body model we use has been developed at EPFL and is depicted by Figure 1(b,c,d,e). It incorporates a highly effective multi-layered approach for constructing and animating realistic human bodies. Ellipsoidal metaballs are used to simulate the overall behavior of bone, muscle, and fat tissue; they are attached to the skeleton and arranged in an anatomically-based approximation. Skin construction is a three step process: First, the implicit surface resulting from the combination of the metaballs influence is automatically sampled along cross-sections. Second, the sampled points become control points of a Bspline patch for each body part. Third, a polygonal surface representation is constructed by tessellating those B-spline patches for seamless joining of different skin pieces and final rendering. This simple and intuitive method combines the advantages of implicit, parametric and polygonal surface representation, producing very realistic and robust body deformations. ∗The work reported here was funded in part by the Swiss National Science Foundation